response model
The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours
Allison, Robert, Maciazek, Tomasz, Stephenson, Anthony
Gaussian process ($GP$) regression is a widely used non-parametric modeling tool, but its cubic complexity in the training size limits its use on massive data sets. A practical remedy is to predict using only the nearest neighbours of each test point, as in Nearest Neighbour Gaussian Process ($NNGP$) regression for geospatial problems and the related scalable $GPnn$ method for more general machine-learning applications. Despite their strong empirical performance, the large-$n$ theory of $NNGP/GPnn$ remains incomplete. We develop a theoretical framework for $NNGP$ and $GPnn$ regression. Under mild regularity assumptions, we derive almost sure pointwise limits for three key predictive criteria: mean squared error ($MSE$), calibration coefficient ($CAL$), and negative log-likelihood ($NLL$). We then study the $L_2$-risk, prove universal consistency, and show that the risk attains Stone's minimax rate $n^{-2α/(2p+d)}$, where $α$ and $p$ capture regularity of the regression problem. We also prove uniform convergence of $MSE$ over compact hyper-parameter sets and show that its derivatives with respect to lengthscale, kernel scale, and noise variance vanish asymptotically, with explicit rates. This explains the observed robustness of $GPnn$ to hyper-parameter tuning. These results provide a rigorous statistical foundation for $NNGP/GPnn$ as a highly scalable and principled alternative to full $GP$ models.
- Europe > United Kingdom (0.14)
- North America > United States > New York > New York County > New York City (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- (2 more...)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- North America > United States > California > Alameda County > Berkeley (0.04)
- Asia > China (0.04)
- (3 more...)
- North America > United States > New York > Tompkins County > Ithaca (0.05)
- North America > Canada (0.05)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.04)
On the Optimality of Tracking Fisher Information in Adaptive Testing with Stochastic Binary Responses
Kim, Sanghwa, Ahn, Dohyun, Min, Seungki
Adaptive testing and sequential estimation problems have recently gained substantial attention due to their foundational role in modern artificial intelligence and interactive systems. Prominent applications include online preference learning, where systems dynamically adapt to user feedback to refine personalized recommendations, and reinforcement learning from human feedback (RLHF), which aims to align AI agents with human values by adaptively querying users. In these contexts, the main focus is to efficiently extract maximal information from human responses, which are inherently stochastic and limited in quantity. Among various types of such problems, this work particularly considers a fundamental yet illustrative case involving stochastic binary responses. Here, a decision-maker sequentially selects questions of varying difficulty from a continuous pool to pose to a candidate and aims to efficiently estimate the candidate's ability (represented by an unknown continuous parameter) by utilizing the binary feedback (e.g., correct/incorrect) collected, which depends probabilistically on the candidate's ability and the question's difficulty. This setup is arguably the simplest scenario that captures the essence of continuous parameter estimation under uncertainty, making it an ideal benchmark for developing fundamental theoretical insights and practical algorithms. Variants of this fundamental adaptive estimation problem have been studied in several communities.
- Asia > South Korea > Seoul > Seoul (0.04)
- Asia > China > Hong Kong (0.04)
- North America > United States > New Jersey > Middlesex County > Piscataway (0.04)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Uncertainty (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.54)
- Information Technology > Artificial Intelligence > Machine Learning > Learning Graphical Models > Directed Networks > Bayesian Learning (0.45)
A Reduced-Dimension fMRI Shared Response Model
Po-Hsuan (Cameron) Chen, Janice Chen, Yaara Yeshurun, Uri Hasson, James Haxby, Peter J. Ramadge
Multi-subject fMRI data is critical for evaluating the generality and validity of findings across subjects, and its effective utilization helps improve analysis sensitivity. We develop a shared response model for aggregating multi-subject fMRI data that accounts for different functional topographies among anatomically aligned datasets. Our model demonstrates improved sensitivity in identifying a shared response for a variety of datasets and anatomical brain regions of interest. Furthermore, by removing the identified shared response, it allows improved detection of group differences. The ability to identify what is shared and what is not shared opens the model to a wide range of multi-subject fMRI studies.
- Health & Medicine > Therapeutic Area > Neurology (1.00)
- Health & Medicine > Health Care Technology (1.00)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- North America > United States > California > Alameda County > Berkeley (0.04)
- Asia > China (0.04)
- (3 more...)
Supplementary Information Neural encoding with visual attention
Mean correlation values across the synchronous, (i.e., stimulus-driven) cortex defined at a range of synchrony thresholds ([0.15,0.75]). We employed the HCP MMP parcellation for all ROI-level analysis. Figure 3 depicts the center-weighted saliency map used in all center-weighted attention models. Thus, the output is a 160900-D vector corresponding to the fMRI response. FDR corrected) for each method are colored on the surface.
- North America > United States > New York > Tompkins County > Ithaca (0.05)
- North America > Canada (0.05)
- North America > United States > New York > Tompkins County > Ithaca (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- North America > Canada (0.04)
- Research Report > Experimental Study (0.93)
- Research Report > New Finding (0.68)
Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models
Nakamura, Mason, Mahmud, Saaduddin, Wray, Kyle H., Zamani, Hamed, Zilberstein, Shlomo
Aligning LLMs with user preferences is crucial for real-world use but often requires costly fine-tuning or expensive inference, forcing trade-offs between alignment quality and computational cost. Existing inference-time methods typically ignore this balance, focusing solely on the optimized policy's performance. We propose HIA (Heuristic-Guided Inference-time Alignment), a tuning-free, black-box-compatible approach that uses a lightweight prompt optimizer, heuristic reward models, and two-stage filtering to reduce inference calls while preserving alignment quality. On real-world prompt datasets, HelpSteer and ComPRed, HIA outperforms best-of-N sampling, beam search, and greedy search baselines in multi-objective, goal-conditioned tasks under the same inference budget. We also find that HIA is effective under low-inference budgets with as little as one or two response queries, offering a practical solution for scalable, personalized LLM deployment.
- North America > United States > Massachusetts > Hampshire County > Amherst (0.04)
- Asia > Middle East > Jordan (0.04)
- Research Report (0.64)
- Workflow (0.46)